How to Solve Numerically the Equilibrium Equations of a Markov Chain with Infinitely Many States

This paper presents a simple and pract ical method to solve the equilïbrium equations of a Markov chain when the number of s ta tes is inf i n i t e . The basic idea is to reduce the inf ini te system of l inear equations to a f in i t e system using the geometrie ta i l behavior of the equil ibrium probabil i t ies . Conditions a re given under which such a reduction is possible. The reduction leads to a remarkably small system of l inear equations which can be rout inely solved by a Gaussian elimination method. An application is given to the D/G/ l queue with scheduled ar r iva ls . How to solve numerically the equilibrium equations of a Markov chain is an important question that arises in numerous problems in applied probability, computer science and operations research. In many of these applications the state space of the Markov chain is infinite. What one usually does to solve numerically the infinite set of equilibrium equations is approximate the infinite Markov model by a truncated model with a finite number of states so that the equilibrium probability of the set of deleted states is very small. Indeed, for a finite-state truncation with a sufficiently large number of states, the difference between the two models will be negligible. However, such a truncation usually leads to a finite but very large system of linear equations whose numerical solution will be quite time-consuming, though an arsenal of good methods are available to solve the equilibrium equations of a finite Markov chain, see [3], [4], [5], [8] and references therein. Moreover, it is somewhat disconcerting that we need a brute-force approximation to solve numerically the infinite-state model. Usually we introducé infinite-state models to obtain mathematical simplification, and now in its numerical analysis using a brute-force truncation we are proceeding in the reverse direction.